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FINAL  REPORT 

BEHAVIORAL  DYNAMICS  IN  THE  COOPERATIVE  CONTROL  OE  MIXED 

HUMAN/ROBOT  TEAMS 

ODDR&E  MURI07,  Program  Grant  Number 
PA9550-07-1-0528 

John  Baillieul 

Intelligent  Mechatronics  Laboratory 
Boston  University 

Eraneeseo  Bullo,  David  Castandn,  Jonathan  D.  Cohen,  Philip  Holmes,  Naomi  Leonard,  Kristi 

Morgansen,  Debora  Prentice,  Juris  Vagners 


Abstract 

This  report  doeuments  researeh  supported  by  the  U.S.  Department  of  Defense,  DDR&E  MURI 
Topie  16,  EY  2007  as  announeed  in  ONR  BAA  06-028.  The  period  of  performanee  was  July  1, 
2007  through  June  30,  2012.  Results  are  reported  on  new  methods  to  eapture,  model,  represent, 
and  ultimately  understand  human  behavior  in  eomplex  and  possibly  adversarial  scenarios 
involving  autonomous  and  semi-autonomous  machine  agents.  Prineiples  and  models  of 
eognitive  and  soeial  psyehology  play  a  major  role  in  the  work.  A  particular  objective  is  to 
develop  a  fundamental  understanding  of  how  humans  and  autonomous  maehine  agents  ean 
operate  as  teams  to  effieiently  aeeomplish  mission  objeetives.  The  researeh  is  foeused  on 
eonditions  in  human-maehine  interactions  in  which  humans  may  not  perform  well  beeause  of 
exeessive  workload,  inability  to  assimilate  information,  and  various  types  of  eognitive  fatigue 
sueh  as  boredom  and  deereased  attentiveness  due  to  task  repetitiveness.  An  equally  important 
foeus  of  the  researeh  is  on  how  human  behavior  differs  from  ideal  deeision  makers  due  to  soeial 
faetors  including  pressure  to  conform,  competitiveness,  and  other  aspeets  of  group  dynamies.  In 
addition  to  exploring  cognitive  and  soeial  psyehological  aspects  of  decision  making,  research  is 
focused  on  formal  approaehes  to  eommunieation  through  aetion.  Understanding  how  gestures 
and  struetured  motions  ean  be  used  to  eommunieate  is  essential  to  involving  mobile  smart 
machines  as  team  members.  An  overarehing  goal  is  an  understanding  of  how  nuaneed  ehanges 
in  colleetive  motions  of  group  of  mobile  agent  serve  to  signal  intentions  of  actions  to  come.  The 
third  major  eomponent  of  the  researeh  being  reported  diseusses  new  results  in  task  partitioning 
between  humans  and  maehines.  The  three  researeh  foei  support  a  paradigm  shift  that  supports 
the  study  of  teams  in  which  humans  operate  on  parity  with  automatons. 


Project  Results  Summary 

The  three  main  eomponents  of  research  supported  by  the  grant  are  (/)  deeision  modeling,  (//) 
aetion-mediated  eommunieation,  and  {in)  task  partitioning.  Progress  on  all  three  of  these  has 
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been  reported  in  workshops,  teehnical  eonferenees,  and  research  publications.  Synergistic 
research  at  the  four  partieipating  institutions  has  sought  to  develop  models  of  deeision-making 
tailored  to  human-in-the-loop  eontrol  of  mixed  human/robot  teams.  There  has  been  signifieant 
progress  in  modeling  human  decision  dynamics  using  Ornstein-Uhlenbeck  (drift-diffusion) 
stochastic  differential  equation  models,  and  reeently  the  team  has  been  working  to  develop 
models  of  seareh  behavior  using  hidden  Markov  models  as  well.  The  goal  in  this  researeh  has 
been  to  develop  models  of  relevant  psyehologieal  dynamies  that  ean  be  integrated  with  the  larger 
body  of  researeh  supported  by  the  Dynamies  and  Control  Program  of  AFOSR.  Work  at  the 
University  of  Washington  and  Prineeton  has  shown  that  for  a  simple  two-alternative  foreed 
ehoiee  deeision  game,  people  tend  to  play  strategies  that  minimize  the  varianee  of  a  reward  while 
avoiding  strategies  that  would  have  higher  reward  payout  over  a  period  of  play.  We  have  shown 
that  this  tendeney  to  be  risk  averse  appears  in  the  stoehastic  models  of  human  behavior  as  well. 
At  Boston  University,  results  have  been  reported  on  seareh  strategies  for  mixed  teams  of  humans 
and  sensor-enabled  robots.  Computer  generated  Gaussian  random  fields  have  been  used  to 
eonstruet  synthetie  environments  in  a  eomputer  game  in  whieh  the  players  eonduet 
reeonnaissance  missions  with  the  goal  of  mapping  the  unknown  field.  In  part  this  study  has 
focused  on  studying  alternative  human  roles  in  interaetions  with  eontrol  algorithms  in  mixed 
human/robot  teams.  Drawing  inspiration  from  early  work  by  N.  Leonard  and  eoworkers, 
Baronov  and  Baillieul  have  developed  feedbaek  eontrol  laws  for  sensor-enabled  mobile  robots 
that  can  steer  the  robots  along  paths  that  traee  eurves  of  eonstant  value  of  a  scalar  potential  field. 
Control  laws  for  aseending  and  deseending  a  potential  field  gradient  have  been  developed  as 
well.  By  appropriately  switehing  between  these  two  kinds  of  eontrol,  sensor-enabled  mobile 
robots  ean  map  the  values  of  an  unknown  potential  field  to  any  desired  degree  of  aeouraey.  This 
mapping  provides  a  useful  abstraction  for  the  exploration  of  quantities  that  vary  over  a  planar 
domain  and  that  ean  be  sensed  by  mobile  agents  moving  in  the  domain.  Examples  of  sueh 
quantities  inelude  oeean  temperature,  CO2  eoneentration,  magnetie  fields,  and  so  forth. 

In  the  studies  that  have  been  eondueted.  The  strategies  that  emerge  from  the  ehoiees  of  motion 
sequences  are  classified  in  terms  of  the  speed  with  whieh  information  is  aequired,  the  fidelity 
with  which  the  aequired  information  represents  the  entire  field,  and  the  extent  to  which  all 
critical  level  sets  have  been  approximated.  The  game  thus  reeords  eaeh  player’s  performanee  in 
aequiring  information  about  both  the  topology  and  geometry  of  the  unknown  fields  that  have 
been  randomly  generated.  Results  so  far  indicate  that  human  subjects  consciously  or 
unconsciously  prefer  to  seek  topologieal  (as  opposed  to  metrie)  information  in  their 
reeonnaissanee  missions.  This  work  has  been  reported  in  a  paper  appearing  in  foeused  speeial 
issue  of  the  Proceedings  of  the  IEEE  (Vol.  100:3,  Mareh,  2012,  pp.  776-801. 
DOl:10.1109/JPROC.s01 1.2174101). 

Progress  has  been  made  in  our  study  of  eommunieation  through  aetion.  We  have  developed 
formal  protoeols  for  eommunieation  based  on  gestures  and  modulating  the  relative  motion 
between  mobile  agents.  This  type  of  eommunieation  is  dependent  on  eontext  in  that  a  particular 
motion  or  gesture  will  indieate  something  related  to  the  current  activity.  The  focus  of  the  work  is 
on  enabling  motion-based  signaling  between  mobile  agents  that  use  motions  and  gestures  that  are 
multiplexed  with  mission-eritieal  motions.  Classes  of  motion  eontrol  laws  that  enables  sueh 
signaling  have  been  developed  and  analyzed.  Properties  of  signals  that  are  amenable  to  sueh 
signaling  have  been  studied,  and  we  have  found  error  bounds  on  the  sensing  of  sueh  motion 
based  signaling.  Current  work  involves  developing  the  foundations  of  multiagent  aetion- 
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mediated  eontrol  using  new  eoneepts  in  what  has  been  ealled  control  communication  complexity. 
In  his  seminal  paper,  “Control  Communieation  Complexity  of  Distributed  Control  Systems,” 
48:3  pp.~1722— 1742,  2009,  W.S.  Wong  extended  A.C.  Yao’s  eoneept  of  eommunieation 
complexity  in  a  very  natural  way  to  a  setting  in  whieh  agents  eommunieate  with  each  other  by 
collaboratively  operating  a  digital  finite  communieation  bandwidth  (DFCB)  control  system. 
Researeh  over  the  past  year  under  the  subjeet  projeet  has  been  aimed  at  exploring  similar  ideas  in 
the  setting  of  time  invariant  linear  eontrol  systems  and  in  the  ease  of  an  broad  and  important 
elass  of  nonlinear  systems  with  elasses  of  standard  inputs. 

The  team  has  made  progress  in  its  multidiseiplinary  effort  to  better  understand  soeial  influenee  in 
deeision-making  for  groups  of  humans.  Analytie  predietions  of  steady-state  performanee  were 
derived  for  multiple  deeision  makers  in  a  two-alternative  foreed  ehoiee  task  with  soeial  feedbaek. 
An  experiment  was  designed  to  test  predictions  with  a  directed  interaetion  network  and  with  both 
human  and  maehine  deeision  makers.  A  seeond  experiment  has  been  eompleted  whieh  examines 
the  separable  influenees  of  information  from  a  spatial  environment  and  from  other  agents. 
Preliminary  analyses  of  behavioral  and  neuroimaging  data  has  yielded  promising  results. 
Bayesian  models  have  been  developed  to  deseribe  how  subjects  learn,  exploit  and  explore  in  this 
experiment. 

We  eonsidered  the  problem  of  sensor  seleetion  for  time-optimal  binary  deteetion.  We  have 
studied  a  group  of  sensors  sequentially  transmitting  their  observations  to  a  fusion  eenter.  The 
fusion  eenter  eonsiders  the  output  of  only  one  sensor  at  the  time.  For  three  distinct  performance 
metrics,  we  have  shown  that  the  fusion  center  almost  surely  needs  to  look  at  at-most  two  sensors. 
In  ongoing  work,  we  are  eonsidering  a  number  of  generalizations  to  multi-hypothesis  testing 
problems,  quiekest  ehange  deteetion  and  distributed  arehiteetures  without  fusion  eenters. 

The  sensor  seleetion  task  was  broken  down  into  three  inter-related  topics.  First,  we  eonsidered 
sensor  seleetion  problems  for  time-optimal  binary  deteetion;  for  a  variety  of  distinct  performance 
metries,  we  showed  that  it  suffiees  to  foeus  attention  on  a  subset  of  sensors.  We  reeently 
developed  (1)  an  adaptive  seheme  that  is  provably  optimal  in  noisy  situations,  and  (2)  a  vehiele 
routing  algorithm  that  exploits  our  sensor  selection  scheme  together  with  eonvex  optimization 
ideas  to  design  UAV  surveillanee  strategies.  Seeond,  we  developed  a  eomprehensive  analysis  of 
eooperative  deeision  making  in  fully  distributed  settings  without  any  leader.  Results  ineluded  the 
eomplete  analysis  of  "sequential  deeision  aggregation"  when  no  interaetions  oeeur  between  the 
agents,  and  this  led  to  the  development  of  a  novel  tractable  model  for  the  ease  in  which  agents 
are  subjeet  to  soeial  peer  pressure.  Third,  as  an  example  of  distributed  decision  making  we 
developed  distributed  algorithms  on  souree  loealization  via  partitioning. 
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Francesco  Bullo  to  deliver  plenary  address  at  the  SIAM  Conf.  on  Control  &  Its  AppL,  Baltimore, 
MD,  Jul  2011 

-  keynote  address  at  2nd  IFAC  Workshop  on  Distributed  Estimation  and  Control  in 
Networked  Systems,  Annecy,  France,  Sep  2010 

Phil  Holmes  was  Opening  Plenary  Speaker,  SIAM  Conference  on  Life  Sciences,  Pittsburgh,  PA, 
July  12-15,  2010 

-  invited  talk  at  the  International  Congress  on  Industrial  and  Applied  Mathematics, 
Vancouver,  BC,  July  18-22,  2011 

John  Baillieul  was  keynote  speaker  “Information  Based  Control  and  Control  Communication 
Complexity,”  2011  Asian  Control  Conference,  Kaohsiung,  Taiwan,  May  17,  201 1 


Transitions 

•  CDC  2008,  December  9-11,  Cancun,  MX:  Invited  Session:  Mixed  Robot/Human  Team 
Decision  Dynamics 

•  ACC  2010,  June  30  -  July  2,  Baltimore,  MD:  Invited  Session:  Humans-In-The-Loop 
Control  Systems 

•  CDC  2010,  December  15-17,  Atlanta,  GA:  Invited  Session:  The  control  theory  of  robot- 
assisted  exploration  of  scalar  potential  fields 

•  NSF  CMMI  Workshop  on  Neuromechanical  Engineering,  September  14-15:  Baillieul 
delivered  one  of  five  keynote  talks:  Cognitive  Aspects  of  Simple  Decisions  in  Search  and 
Reconnaissance 
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Baillieul  et  al. 


•  United  Teehnologies  Researeh  Center,  August  17,  2010,  The  Boston  University  team  will 
have  a  daylong  visit  with  UTRC  researeh  staff  to  diseuss  researeh  supported  by  the 
MURI 

•  Speeial  Issue  of  the  Proeeedings  of  the  IEEE  on  Interaction  Dynamics  at  the  Interface  of 
Humans  and  Smart  Maehines,  Eds:  Baillieul,  Leonard,  Morgansen.  Publieation  target: 
November,  2011 

•  Keynote  talk  “Information  Based  Control  and  Control  Communieation  Complexity,” 
2011  Asian  Control  Conference,  Kaohsiung,  Taiwan,  May  17,  201 1 

•  2011  Control  Systems  Soeiety  Bode  Lecture,  “Eifty  Years  of  Information  Based  Control 
Theory,”  To  be  delivered  by  John  Baillieul  at  the  50*  IEEE  Conferenee  on  Deeision  and 
Control,  Deeember,  2011. 


